Contrastive Learning with Prompt-derived Virtual Semantic Prototypes for Unsupervised Sentence Embedding

Contrastive learning has become a new paradigm for unsupervised sentence embeddings. Previous studies focus on instance-wise contrastive learning, attempting to construct positive pairs with textual data augmentation. In this paper, we propose a novel Contrastive learning method with Prompt-derived Virtual semantic Prototypes (ConPVP). Specifically, with the help of prompts, we construct virtual semantic prototypes to each instance, and derive negative prototypes by using the negative form of the prompts. Using a prototypical contrastive loss, we enforce the anchor sentence embedding to be close to its corresponding semantic prototypes, and far apart from the negative prototypes as well as the prototypes of other sentences. Extensive experimental results on semantic textual similarity, transfer, and clustering tasks demonstrate the effectiveness of our proposed model compared to strong baselines. Code is available at https://github.com/lemon0830/promptCSE.Comment: Findings of EMNLP 202

Farthest Greedy Path Sampling for Two-shot Recommender Search

Weight-sharing Neural Architecture Search (WS-NAS) provides an efficient mechanism for developing end-to-end deep recommender models. However, in complex search spaces, distinguishing between superior and inferior architectures (or paths) is challenging. This challenge is compounded by the limited coverage of the supernet and the co-adaptation of subnet weights, which restricts the exploration and exploitation capabilities inherent to weight-sharing mechanisms. To address these challenges, we introduce Farthest Greedy Path Sampling (FGPS), a new path sampling strategy that balances path quality and diversity. FGPS enhances path diversity to facilitate more comprehensive supernet exploration, while emphasizing path quality to ensure the effective identification and utilization of promising architectures. By incorporating FGPS into a Two-shot NAS (TS-NAS) framework, we derive high-performance architectures. Evaluations on three Click-Through Rate (CTR) prediction benchmarks demonstrate that our approach consistently achieves superior results, outperforming both manually designed and most NAS-based models.Comment: 9 pages, 5 figure

Cross-Architecture Knowledge Distillation

Transformer attracts much attention because of its ability to learn global relations and superior performance. In order to achieve higher performance, it is natural to distill complementary knowledge from Transformer to convolutional neural network (CNN). However, most existing knowledge distillation methods only consider homologous-architecture distillation, such as distilling knowledge from CNN to CNN. They may not be suitable when applying to cross-architecture scenarios, such as from Transformer to CNN. To deal with this problem, a novel cross-architecture knowledge distillation method is proposed. Specifically, instead of directly mimicking output/intermediate features of the teacher, a partially cross attention projector and a group-wise linear projector are introduced to align the student features with the teacher's in two projected feature spaces. And a multi-view robust training scheme is further presented to improve the robustness and stability of the framework. Extensive experiments show that the proposed method outperforms 14 state-of-the-arts on both small-scale and large-scale datasets

Soft Language Clustering for Multilingual Model Pre-training

Multilingual pre-trained language models have demonstrated impressive (zero-shot) cross-lingual transfer abilities, however, their performance is hindered when the target language has distant typology from source languages or when pre-training data is limited in size. In this paper, we propose XLM-P, which contextually retrieves prompts as flexible guidance for encoding instances conditionally. Our XLM-P enables (1) lightweight modeling of language-invariant and language-specific knowledge across languages, and (2) easy integration with other multilingual pre-training methods. On the tasks of XTREME including text classification, sequence labeling, question answering, and sentence retrieval, both base- and large-size language models pre-trained with our proposed method exhibit consistent performance improvement. Furthermore, it provides substantial advantages for low-resource languages in unsupervised sentence retrieval and for target languages that differ greatly from the source language in cross-lingual transfer

Interrelation between the lipid accumulation product index and diabetic kidney disease in patients with type 2 diabetes mellitus

ObjectiveThe purpose of this study was to determine the relation between the lipid accumulation product index (LAPI) and diabetic kidney disease (DKD) in patients with type 2 diabetes mellitus (T2DM).MethodsHerein, 931 patients were enrolled and their data were collected. Then the interrelation between LAPI and DKD was assessed using multivariate logistic regression analyses (LRAs) and by a restricted cubic spline (RCS).ResultsIn total, 931 participants (352 females and 579 males) aged 55 years on average were included in the study. After adjusting for several confounders, the odds ratio for DKD was increased evidently in the third LAPI tertile compared with that in the first LAPI tertile. In addition, the RCS revealed a positive interrelation between LAPI and DKD. In the subgroup analyses, age, sex, hyperlipidemia, hypertension, and HbA1c did not significantly interact with LAPI.ConclusionsLAPI was higher in the DKD group than in the no-DKD group, and LAPI is positively linked with DKD, which may have potential value to diagnose DKD in clinical practice

Evaluation of Lethal Giant Larvae as a Schistosomiasis Vaccine Candidate

Schistosomiasis is a neglected tropical disease of humans, and it is considered to be the second most devastating parasitic disease after malaria. Eggs produced by normally developed female worms are important in the transmission of the parasite, and they responsible for the pathogenesis of schistosomiasis. The tumor suppressor gene lethal giant larvae (lgl) has an essential function in establishing apical-basal cell polarity, cell proliferation, differentiation, and tissue organization. In our earlier study, downregulation of the lgl gene induced a significant reduction in the egg hatching rate of Schistosoma japonicum (Sj) eggs. In this study, the Sjlgl gene was used as a vaccine candidate against schistosomiasis, and vaccination achieved and maintained a stable reduction of the egg hatching rate, which is consistent with previous studies, in addition to reducing the worm burden and liver egg burden in some trials

Result of a year-long animal survey in a state-owned forest farm in Beijing, China

BackgroundArtificial forest can have great potential in serving as habitat to wildlife, depending on different management methods. As the state-owned forest farms now play a new role in ecological conservation in China, the biological richness of this kind of land-use type is understudied. Once owned by a mining company, a largest state-owned forest farm, Jingxi Forest Farm, has been reformed to be a state-owned forest farm with the purpose of conservation since 2017. Although this 116.4 km2 forest farm holds a near-healthy montaine ecosystem very representative in North China, a large proportion of artificial coniferous forest in the forest farm has been proven to hold less biodiversity than natural vegetation. This situation, however, provides a great opportunity for ecological restoration and biodiversity conservation. Therefore, from November 2019 to December 2020, we conducted a set of biodiversity surveys, whose results will serve as a baseline for further restoration and conservation.New informationHere, we report the result of a multi-taxa fauna diversity survey conducted in Jingxi Forest Farm mainly in year 2020 with explicit spatial information. It is the first survey of its kind conducted in this area, revealing a total of 19 species of mammals, 86 birds, four reptiles, two amphibians and one fish species, as well as 101 species of insects. Four species of mammals are identified as data-poor species as they have less than 100 occurrence records with coordination in the GBIF database. One species of insect, representing one new provincial record genus of Beijing, is reported

Manipulating Protein Conformations By Single-molecule Afm-fret Nanoscopy

Combining atomic force microscopy and fluorescence resonance energy transfer spectroscopy (AFM-FRET), we have developed a single-molecule AFM-FRET nanoscopy approach capable of effectively pinpointing and mechanically manipulating a targeted dye-labeled single protein in a large sampling area and simultaneously monitoring the conformational changes of the targeted protein by recording single-molecule FRET time trajectories. We have further demonstrated an application of using this nanoscopy on manipulation of single-molecule protein conformation and simultaneous single-molecule FRET measurement of a Cy3-Cy5-labeled kinase enzyme, HPPK (6-hydroxymethyl-7,8-dihydropterin pyrophosphokinase). By analyzing time-resolved FRET trajectories and correlated AFM force pulling curves of the targeted single-molecule enzyme, we are able to observe the protein conformational changes of a specific coordination by AFM mechanic force pulling

Potential for a Circular Autopoietic Economy on Canavese Territory

The aim of this gigamap is to describe and summarize a student work carried out during the semester course Open Systems Design at Politecnico Torino. The map is one of the outputs from an analysis of the Canavese, Piedmont region and subsequently an in-depth study of the relationships and “flows” with certain “currencies” between some selected economic and public actors. Canavese is a historical-geographical area located in the province of Turin, Italy; it’s full of great natural and geological spots, with a strong enogastronomic identity, historical re-enactments, a well-known manufacture sector like the ceramic of Castellamonte, as well as the leading high-tech branch like Arduino. Through the approach deriving from systemic design, it was possible to identify some strategies for the creation of new networks aimed at transforming the economy and production processes from linear to circular approach. Five types of “circular flows” were assessed and “designed into” the current linear system, thus proposing an improved economy based on circularity: flows of matter (energy/CO2, water, materials), economic flows, and social flows. The Gigamap will be presented to stakeholders in a public hearing and supposedly be used to illustrate and incubate a circular economy that is more resilient and more regenerative. Reading the map The reading should start at the top left, where the abstract is located, next, there is an outline that shows the connections between the five topics investigated. Subsequently, there is a timeline with some fundamental events in the history of the Canavese. In the center there is the circularity map with the main cities, focusing on new opportunities of circular flows between the 15 analysed economic actors. At the bottom, some general data and on the right a brief description of the role and territorial qualities of the companies. Lastly, there is a summary of the emerging properties

Optimal design of a non-isolated high power density buck-boost converter

Semiconductor lifetime and power density are considered to be two important development directions of power converter design in the future. Especially in some high dynamic application, the operation condition is changing frequently causing the temperature swing of semiconductor, and it leads to thermal expansion and contraction which affects the lifetime or reliability of semiconductor. A high maximum temperature or a large temperature swing leads to short lifetime. Meanwhile, high power density is a general trend. If the size of a whole system is limited, a high power density converter enables more functionality of the system. Also, high power density also leads to high portability. Works have been done to use interleaved and high switching frequency converter to get a high power density. However, those converters such as interleaved buck converter do not help to improve semiconductor reliability because it can only operate in synchronous conduction mode (SCM) and triangular current mode (TCM).In this work, a $20kW$ DC/DC converter for highly dynamic application is designed. The four-switch buck-boost converter is chosen because of its flexibility and its possibility to get longer semiconductor lifetime and higher power density. It is possible to offer a free-wheeling interval in which no voltage is applied on the inductor, the transferred power can be regulated by adjusting the duty cycle of this interval. In addition, an optimal modulation scheme is proposed for this topology which further helps improve the reliability of semiconductor. An optimal power stage design is given which is a two-stage interleaved structure. Plus, a close-loop control strategy is given according to the design.The proposed modulation scheme is tested with a low voltage level prototype, and the performance prediction is verified